1. Field
Embodiments disclosed herein are directed to a system that co-schedules network resource provisioning and host-to-host bandwidth reservation on high-performance network and storage systems.
2. Description of Related Art
Data-intensive application communities, including high energy and nuclear physics, astrophysics, climate modeling, nanoscale materials science, and genomics are expected to generate exabytes of data over the next five years. Such data must be transferred, analyzed, and visualized by geographically distributed teams of scientists. This expectation of explosive growth in stored data and globally distributed data processing needs, underpinned by the maturing grid and cloud computing technologies, has generated critical requirements for new predictable and well-behaved data transfer technologies and automated tools. To expedite scientific discoveries, these data transfer tools need to intelligently assist scientists in replicating large volumes of data whenever and wherever necessary.
Existing data transfer techniques face unprecedented challenges in handling not only the volume of data, but also the heterogeneous environment where data are imported and exported. An obstacle to managing these challenges is the inability to provide end-to-end bandwidth guarantees from source storage systems to destination storage systems. Further, technology advancements give rise to performance improvements while also increasing the complexity of resource management and provisioning. Data storage technologies have demonstrated significant improvements through the use of advanced parallel file systems that enhance I/O bandwidth, and solid state disks (SSD) that can provide read/write access as much as ten times faster than hard drives.